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Abstract 



The statistical physics approach to the number partioning problem, a classical NP- 
hard problem, is both simple and rewarding. Very basic notions and methods from 
statistical mechanics are enough to obtain analytical results for the phase boundary 
that separates the "easy-to-solve" from the "hard-to-solve" phase of the NPP as 
well as for the probability distributions of the optimal and sub-optimal solutions. 



In addition, it can be shown that solving a number partioning problem of size N 
to some extent corresponds to locating the minimum in an unsorted list of 0(2 N ) 
numbers. Considering this correspondence it is not surprising that known heuristics 
for the partitioning problem are not significantly better than simple random search. 
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1 Introduction 



Recent years have witnessed an increasing interaction among the disciplines of 
discrete mathematics, computer science, and statistical physics. These fields 
are linked by the fact that models from statistical physics can be formalized as 
combinatorial optimization problems and vice versa [28,30]. The connection 
between optimization and statistical physics has lead to practical algorithms 
like simulated annealing [19] and to new theoretical results, some of which can 
be found in this special issue. 

In most cases where a statistical physics analysis of an optimization or decision 
problem yields significant new results, this analysis is rather complicated, tech- 
nically as well as conceptionally. This complexity may easily deter computer 
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science people from learning the tricks and tools, even if they value the results. 
To promote interdisciplinarity beyond the mutual appreciation of results, it 
may help to consider a physicists approach to an optimization problem, which 
on the one hand requires only very basic notions and methods from statisti- 
cal mechanics, but on the other hand yields non trivial results. In fact there 
exists a problem with this nice property: the number partitioning problem. It 
is defined as follows: Given a list 01,02,... , Ojv of positive numbers, find a 
partition, i.e. a subset A C {1, . . . , N} such that the partition difference 
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is minimized. In the constrained partition problem, the cardinality difference 
between A and its complement, 

M = \A\ - (N - \A\) = 2\A\ - N, (2) 

is fixed. A special case is the balanced partitioning problem with the constraint 
\M\ < 1. 

Partitioning is of both theoretical and practical importance. It is one of Garey 
and Johnson's six basic NP-complete problems that lie at the heart of the 
theory of NP-completeness [10]. Among the many practical applications one 
finds multiprocessor scheduling and the minimization of VLSI circuit size and 
delay [4,33]. 

In this paper we present a statistical mechanics approach to the NPP. We 
start, however, with a brief discussion of some known facts about the NPP. 
We will learn that there is a phase transition in the computational complexity 
of the NPP, and that there are no really good heuristics for this problem. 
Both facts will be discussed within the framework of statistical mechanics in 
the following sections. Section 3 starts with an introduction into the very basic 
notions and methods of statistical mechanics. We formulate the NPP as a spin- 
glass, i.e. as a model to describe magnetic alloys, and calculate its free energy 
and entropy. The entropy in turn yields a simple analytic expression for the 
phase boundary that separates the "easy-to-solve" from the "hard-to-solve" 
phase in the NPP. In addition, we get an expression for the average optimum 
partition difference. The statistical mechanics analysis reveals another phase 
transition in the constrained NPP: if M exceeds a critical value, the NPP 
becomes overconstrained and its solution trivial. In section 4 we map the 
balanced and the unconstrained NPP to another physical model, the random 
energy model. This signifies that solving the NPP with N random numbers 
aj corresponds to locating the minimum in an unsorted list of 0(2 N ) random 
numbers. This correspondence provides us with an explanation of the bad 
performance of heuristic algorithms for the NPP and in addition allows us to 
derive analytical expressions for the probability distribution of the optimal and 



sub-optimal costs. 



2 Some facts about number partitioning 



The computational complexity of the number partitioning problem depends 
on the type of input numbers {01,02,... , oat}. Consider the case that the 
Oj's are positive integers bound by a constant A. Then the cost E can take 
on at most NA different values, i.e. the size of the search space is O(NA) 
instead of 0{2 N ) and it is very easy to devise an algorithm that explores 
this reduced search space in time polynomial in NA. Unfortunately, such an 
algorithm does not prove P = NP since a concise encoding of an instance 
requires 0(N log A) bits, and A is not bounded by any polynomial of log A 
This feature of the NPP is called "pseudo polynomiality" . The NP-hardness 
of the NPP requires input numbers of arbitrary size or, after division by the 
maximal input number, of unlimited precision. 

To study typical properties of the NPP, the input numbers are usually taken to 
be independently and identically distributed (i.i.d.) random numbers, drawn 
from "well behaved" distributions. Under this probabilistic assumption, the 
minimal partition difference E\ is a stochastic variable. For real valued input 
numbers (infinite precision, see above), Karmarkar, Karp, Lueker and Odlyzko 
[18] have proven that the median value of E 1 is 0(yN ■ 2~ N ) for the uncon- 
strained and 0(N ■ 2~ N ) for the balanced NPP. Lueker [23] showed recently, 
that the same results hold for the average value of E\. Numerical simulations 
[7] indicate, that the relative width of the distribution of E±, defined as 
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where (•) denotes the average over the a/s, tends to 1 in the limit iV — > oo, for 
both the unconstrained and the balanced partitioning problem. This means, 
that the typical fluctuations of E\ are of the same size than the value itself. 
In section 4 we will calculate the complete probablity distribution of E\ and 
rederive all these results. 

Another surprising feature of the NPP is the poor performance of heuristic 
algorithms [16,32]. In section 4 we show that the bad efficiency of heuristics 
approaches can be understood by the observation that number partitioning is 
essentially equivalent to locating the minimum in an unsorted list of 0(2 N ) 
random numbers [27]. Here we will describe some of the heuristics. 

The key ingredient to the most powerful partition heuristics is the differencing 
operation [17]: select two elements a,i and a,- and replace them by the element 



I Oj — Oj I . Replacing o« and Oj by | Oj — Oj | is equivalent to making the decision 
that they will go into opposite subsets. Applying differencing operations N — 1 
times produces in effect a partition of the set {ai, . . . ,a^}. The value of 
its partition difference is equal to the single element left in the list. Various 
partitions can be obtained by choosing different methods for selecting the 
pairs of elements to operate on. In the paired differencing method (PDM), the 
elements are ordered. The first |_7V/2J operations are performed on the largest 
two elements, the third and the fourth largest, etc.. After these operations, the 
left-over |~iV/2] elements are ordered and the procedure is iterated until there 
is only one element left. Another example is the Karmarkar-Karp (KK) or 
largest differencing method [17]. Again the elements are ordered. The largest 
two elements are picked for differencing. The resulting set is ordered and the 
algorithm is iterated until there is only one element left. The time complexity 
of PDM and KK is 0(N log N), the space-complexity is 0(N). 

The Karmarkar-Karp differencing is the best known heuristics for the par- 
tioning problem, but it finds an approximate solution only, far away from 
the true optimum. KK yields unconstrained partitions with expected differ- 
ence 0(N~ alogN ), which has to be compared to 0(y~N ■ 2~ N ) for the true 
optimum. Korf [21] showed, how the KK differencing can be extended to a 
complete anytime algorithm, i.e. an algorithm that finds better and better so- 
lutions the longer it is allowed to run, until it finally finds and proves the 
optimum solution: At each iteration, the KK heuristic commits to placing the 
two largest numbers in different subsets, by replacing them with their differ- 
ence. The only other option is to place them in the same subset, replacing 
them by their sum. This results in a binary tree, where each node replaces 
the two largest remaining numbers, a\ > a^. the left branch replaces them by 
their difference, while the right branch replaces them by their sum: 

|oi — 02I, 03, • • ■ left branch 
ai,a 2 ,o 3 , ... h-> { (4) 

ai + a 2 ,a 3 , . . . right branch 

Iterating both operations TV — 1 times generates a tree with 2 Ar_1 terminal 
nodes. The terminal nodes are single element lists, whose elements are the valid 
partition differences. Korf's complete Karmarkar-Karp differencing algorithm 
(CKK) searches this tree depth-first and from left to right. The algorithm first 
returns the KK-heuristic solution, then continues to find better solutions as 
time allows. See Fig. (1) for the example of a tree generated by the CKK. 

There are two ways to prune the tree: At any node, where the difference 
between the largest element in the list and the sum of all other elements is 
larger than the current minimum partition difference, the node's offspring can 
be ignored. For integer valued aj, a partition with E < 1 is called perfect. If 
one reaches a terminal node with a perfect partition, the entire search can be 
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Fig. 1. Tree generated by the CKK algorithm on the list 8,7,6,5,4. Left branch: 
Replace the two largest numbers by their difference. Right branch: Replace the 
two largest numbers by their sum. The dashed parts of the tree are pruned by the 
algorithm. Thanks to the pruning rules, only 9 of 31 nodes have to be explored. 

terminated since no improvement is possible. The dashed nodes in Fig. (1) are 
pruned by these rules. 

A variant of Korf s complete differencing method to solve the balanced NPP is 
called the complete balanced largest differencing method, BLDM [26]. It works 
very similar to the Korf-algorithm but generates only balanced partitions. To 
measure the performance of this algorithm as an exact solver, we count the 
number of nodes generated until the optimum solution has been found and 
proven. The result for 25- bit integers is shown in Fig. (2). Each data point 
represents the average of 100 random problem instances. The horizontal axes 
shows the number of integers to be partitioned, the vertical axes shows the 
number of nodes generated (left) and the fraction of instances that have a 
perfect partition (right). Note that we counted all nodes of the tree, not just 
the terminal nodes. We observe three distinct regimes: for TV < 30, the number 
of nodes grows exponentially with N, for iV > 30 it decreases with increasing 
N, reaching a minimum for N ps 130 and starting to grow like N for larger 
values of N. 



The region of exponential growth is characterized by the lack of perfect par- 
titions. In this regime, the algorithm has to search the whole tree in order to 
find and prove the optimum partition. For N > 30 it finds a perfect partition 
and stops the search prematurely. The number of perfect partitions seems to 
increase with increasing N, making it easier to find one of them. This would 
explain the decrease of search costs. For N 3> 30, the KK partition already is 
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Fig. 2. Number of nodes generated by the complete BLDM algorithm to optimally 
partition random 25-bit integers. The complete BLDM algorithm is a variant of 
Korf's complete differencing method, modified to solve the balanced NPP [26]. 



perfect: The construction of this first partition always requires iV nodes (N — l 
internal nodes and one leaf). 



Numerical simulations show that this a general scenario in the NPP. For i.i.d. 
random 6-bit numbers a,j, the solution time grows exponentially with iV for 
N < b and polynomially for N 3> b [12,20,21]. Problems with N ~ b require 
the longest solution time. The transition from the "hard" to the computational 
"easy" phase has some features of a phase transition in physical systems. Phase 
transitions of this kind have been observed in numerous NP-complete prob- 
lems [3,11,31], and can often be analyzed quantitatively in the framework of 
statistical mechanics. Compared to other problems, this analysis is surpris- 
ingly simple for the number partitioning problem [25]. This is what we will do 
in the next section. 



3 Phase transitions 



The observed transition from a computationally "hard" to an "easy" regime in 
the NPP can well be analysed within the framework of statistical mechanics. 
We start with a very brief and superficial sketch of statistical mechanics and 
how it can be used to study combinatorial optimization, introducing the basic 
quantities free energy and entropy and rewriting the NPP as a physical model 
system, a spin glass. Then we calculate the free energy and the entropy of the 
unconstrained NPP, learning some tools from the physicists toolbox like the 
(^-function and the Laplace method to evaluate integrals. The entropy allows 
us to define a precice expression for the control parameter that fixes whether 
the NPP is "hard" or "easy". After that we try the same calculation for the 
constrained NPP, which is a bit more cumbersome. It turns out that a com- 
plete solution of the general constrained NPP requires some numerics, but the 
balanced NPP can be solved analytically. Again we find the control parameter 
and its critical value. The numerical solution of the general constrained NPP 
reveals the existence of another phasetransition from a computational hard to 
computational easy phase. The control parameter for this phase transition is 
M, the imposed cardinality difference. This phase transition is dicussed in the 
last section. 



3.1 Statistical mechanics, optimization and spin glasses 



The aim of statistical mechanics is to predict the properties of systems com- 
posed of very large numbers of particles in terms of the mechanical properties 
of the individual particles and of the forces between them. How large is "very 
large"? A few gramm of matter consists of about 10 23 atoms. The state of 
a system is specified by the position, velocity, magnetization, ... of each of 
these atoms. The equations that describe the evolution of this microstate in 
time are known in princible, but it is completely hopeless to solve them for 
10 23 particles. From everyday experience we know however, that the temper- 
ature and the pressure of a gas in a vessel do not change in time, although 
the microstate, i.e. the positions and velocities of all the gas atoms, keeps 
changing all the time. Hence the macroscopic properties of a system are not 
sensitive to its particular microscopic state. This physical variant of the law of 
large numbers constitutes the starting point of statistical mechanics: Instead 
of determining the exact value of a macroscopic quantity in a single system, 
its average value is computed, taken over a suitable ensemble of similarly pre- 
pared systems. The ensemble average value is usually much easier to compute 
than the exact value. Note that this works because of the large number N of 
particles. The results of statistical mechanics are valid only in the N — > oo, 
the so called thermodynamic limit. The use of a capital letter N shall remind 



you of this. 

Consider a system with possible microstates s G S. For a gas, s contains the 
positions and velocities of all atoms. If the system is kept at a temperature 
T, according to statistical mechanics macroscopic quantities (like the pres- 
sure) can be calculated as averages over the canonical ensemble, in which each 
microstate has a probability 

p(s) = ±e-*W (5) 

H(s) is termed the Hamiltonian of the system. It is a real valued function that 
yields the energy of the microstate s. The normalization factor 

Z = £ e~ H ^ T . (6) 

seS 

is called the partition function, not to be mixed up with the partition in the 
NPP. The thermal average of a quantity A is given by 

(A) T = W A( S )e~ H ^ T . (7) 

The thermal average no longer depends on a particular microstate, but only 
on macroscopic parameters like the temperature T, reflecting precisely the 
experimental observations. 

The central quantity that is calculated in statistical mechanics, is the free 
energy F, 

F(T) = -T\nZ. (8) 

Once F is known as a function of the temperature T and other relevant param- 
eters like volume or magnetic field, most properties of the system can easily 
be calculated. At least this is what they tell you in textbooks on statistical 
physics. The thermal average of the energy for example is given by 

(H) T = I 5>(*)«T HW/T = T 2 ±F(T). (9) 



What has all this to do with combinatorial optimization? Well, we can formally 
define a free energy for any optimization problem: S is the set of all feasible 
solutions, and H(s) is the cost function that has to be minimized. This free 
energy comprises some usefull information about the optimization problem. 
Let Ei < E 2 < E 3 < . . . be the sorted list of possible values of the cost 
function and n(Ej,) be the number of feasable solutions that yield E = Ej,. 
Then the free energy is 



F(T) = -T\nZ 
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From that we get the value of the optimum solution 

lim F(T) = E x 

as well as the logarithm of the number of optimum solutions 

d 
lim -—F(T) = lim S(T) = lnn(^i). 

In physics jargon, S(T) is the entropy, E\ the ground state energy. By adding 
additional terms to the cost function and recalculating the free energy, more 
informations can be obtained, for instance on the structure of the optimum 
solution. If you do not like all the physics jargon you might consider the 
free energy as a kind of generating function that encodes properties of your 
combinatorial optimization problem. 

A class of models that have been intensely investigated in physics are spin 
glasses [28] . In its simplest form, the microstate of a spin glass is a set of N 
binary variables, Sj = ±1, j = 1, . . . , N, called Ising spins. With spin glass 
models, physicists try to capture the properties of magnetic alloys. An Ising 
spin is the magnetic moment of an atom that can only be oriented along a 
given axis in space, either "up" (sj = +1) or "down" (sj = —1). In an alloy 
these moments interact, giving rise to a total energy 

N 

H({sj}) = -J2 JijSiSj- ( 10 ) 

The Jij are numbers that describe the interaction strength between spins s, 
and Sj. The calculation of the interactions Jy (as well as the justification for 
the whole model) is a subject of quantum mechanics and will not be discussed 
here. 

Minimizing the spin glass hamiltonian for given interactions J^ is a com- 
binatorial optimization problem. If all the J^ are positive (physics jargon: 
ferromagnetic), this problem is trivial: H is minimized when all spins point 
in the same direction, i.e. are all +1 or all — 1. If some (or all) of the J^ are 
negative (physics jargon: anti-ferromagnetic), this problem is much harder. In 
fact it can be proven that it is NP-hard [1]. 



A partition A in the number partitioning problem can be encoded by Ising 
spins: Sj = +1 if j G A, Sj = — 1 otherwise. The cost function then reads 

N 

E=\'£a j 8 j \, (11) 

and the minimum partition is equivalent to the ground state of the Hamilto- 
nian 

N 

H = E 2 = ^2 s i a i a j s j- (12) 

This is an infinite range Ising spin glass with antiferromagnetic couplings 
Jij = —didj < 0. The statistical mechanics of this model has been investi- 
gated in physics at least three times [8,7,25]. It turns out that due to the 
multiplicative character of the couplings, the calculation of the free energy is 
comparatively simple, and yields quantitative results on the phase transition 
in computational complexity [25]. 

Of course we are not interested in a particular instance but in the typical 
properties of number partitioning. Hence we will average our results over a 
suitable ensemble of instances, not to be mixed up with the thermodynamic 
ensemble of microstates resp. feasible solutions. Throughout this paper we 
will assume that the input numbers Oj are independent, identically distributed 
(i.i.d.) random numbers. In our statistical mechanics framework, random input 
numbers correspond to random spin interactions J^. In fact this is part of the 
definition of spin glass models - the term "glass" refers to the irregularity 
of the interactions in alloys as opposed to regular interactions in crystalls. 
In spin glass theory, it is the free energy that has to be averaged over the 
random couplings to yield the correct typical properties of the system. In 
general, the computation of the average free energy is not simple and requires 
a sophisticated approach called the replica method. The free energy of the 
number partitiong problem is so simple though, that we get its average for 
free. 



3.2 Statistical mechanics of the unconstrained NPP 



We start with the statistical mechanics of the unconstrained NPP. This anal- 
ysis has been published elsewhere [25], but the presentation here is more com- 
prehensive. The partition function of the unconstrained NPP reads 

Z=£) e -Tl£j«y«il. (13) 
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Without the absolute value in the exponent, this sum can easily be calculated: 



iV 



{sj} {sj}i =1 

Sl=±l S2=±l Sjv=±1 

= 2 cosh — • 2 cosh — • . . . • 2 cosh — 
T T T 

= 2^ ft cosh ^ (14) 

The question is, how can we get rid of the absolute value in the exponent? 
A standard trick in statistical mechanics to remove nasty nonlinearities like 
this is the creative use of the 5-function. Introduced by P.A.M. Dirac on an 
intuitive base in connection with quantum mechanics, it is now embedded in 
an exact mathemetical framework [22]. Here we stick to the more intuitive 
picture and define the 5-function via its Fourier integral, 

S( x ) = -!- f" dxe ixx . (15) 

Z7T J -oo 

S(x) is for x 7^ and oo for x = 0, and the peak at x = is perfectly 
calibrated to give 

dxf(x)5(x — c) = /(c) (16) 

for any reasonably well behaved function /. The 5-function helps us to separate 
the absolute value from the summation variables Sj\ 



a j S 3, 



/•oo 1 ^ 

Z=W dxe-\*\6(x--52, 

f , J —oo 1 ■ -, 

/OO 1 /"OO . j, s—^ 

dx e _|a:| — / dxe ixx V e~** ^ aj 
-oo 27C J-oo , -, 



Z = 2 N [°° —{[ cos(^x) rdxe-M +lxx . (17) 

J— oo 2/R ■ ! 1 J —oo 



{Sj} 

Now we can carry out the summation over the {sj} like in eq. (14): 

'°° dx N 

3=1 

Note that cosh(ix) = cos(x). Doing the x-integral, 

rl^. p -\x\+ixx _ j 

1 + x 2 ' 
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and substituting y = arctanx finally leads us to 



Z = 2 N f*dy e NG(y) (1Q) 



with 



1 N a 

G(y) = -^\ncos&tan(y)). (20) 

iV 3=1 J 



For large values of N, the statistical independence of the a,j allows us to apply 
the law of large numbers, i.e. to replace the sum by the average over a: 

G(y) n (hico8(ltm(y))y (21) 

This replacement is the main reason why spin glasses with couplings that 
factorize, J^- = — didj, are comparatively easy to solve [29]. It relieves us from 
averaging In Z, which can be very difficult in other spin glass models. 

The integral in eq. (17) can be evaluated asymptotically for large N using 
the Laplace method: The general idea is, that the integral is dominated by the 
contributions from the maxima of G(y). If G(y) has a maximum at y — yo, 



e NG ^dx » e NG ^ J e -TG"(vo)(i/-vo) 2 dy = e NG <*°)J ^ (22) 

for large N . A general discussion of the Laplace method for the asymptotic 
expansion of integrals can be found in various text-books [2,24]. 

To find the maxima of G(y), we will assume that a can only take on values 
that are integer multiples of a fixed number Aa. For integer distributions 
Aa = 1, and for floatingpoint distributions Aa is the smallest number that 
can be represented with the available number of bits. This is a reasonable 
assumption since we know, that the properties of the NPP depend on the 
resolution in a. With this assumption, the solutions of 

G'(y) = ^tan (|tanj/)^ • (1 +tan 2 y) = (23) 



are given by 



/7rT 
y k = arctanf — k) k = 0, ±1, ±2, . . . . (24) 



Note that tan ( ^ tan^J = for all values a = n ■ Aa. Of course we have to 
consider the contributions of all saddle points when evaluating the integral in 
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Eq. (17): 
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and the useful identity 



we finally get 



Y^ -, / , no = — ■ coth — 

-Oil ~*~ l 3 ^) x x 



N Aa , Aa 

Z = 2 N ■ , • coth — . 

T 



yp^y 



(25) 



(26) 



(27) 
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The partition function Z immediately yields the free energy 

F(T) 
and the thermal average of the energy 

(E) 



-TNln2 + -ln 

2 2Aa 2 



Aa 



T ~ sinh 4? cosh 4? ' 



(29) 



(30) 



Let Aa > be fixed. Then lim^^o (E) T = 0, i.e. the ground states are perfect 
partitions. How many perfect partions can we expect? The answer is given by 
the entropy S, which according to eq. (28) and eq. (8) can be written as 



with 



S = N(K C -K)ln2 + S(^] 



K r — 1 " ~ 

N2\n2 



(31) 



(32) 



ln^> 
7V21n2 ' 

and the thermal contribution to the entropy is 



s(p, 



, Aa Aa coth 2 ^ - 1 
lnC ° th ^ + ^ coth-^ ' 
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For finite Aa, S vanishes at zero temperature and increases monotonically 
with T. In this case, the zero temperatur entropy is given by N(k c — k) In 2. If 
k < k c , we have an extensive entropy resp. an exponential number of perfect 
partitions. N ■ k is a measure for the number of bits needed to encode the Oj's. 
Let the cij be i.i.d. 6-bit integer numbers. Then Aa = 1 and 

b 1 ■ /- 3„_ 6 1 2(A 6 1 -..-.jv 



K = TV + 2JV "* i 1 - 2 2 + 2 2 > = N + N- °< 2 >' < 35) 



In this case the condition k < k c translates into 

6<iV-iln 2 (^iV). (36) 

This inequality must be fullfilled in order to have perfect partitions. The first 
term on the right hand side can be explained within a simple approximation 
[12]: Let the N numbers a$ each be represented by b bits. Now consider the 
partition difference E bitwise. About half of all partitions will set the most 
significant bit of E to zero. Among those partitions, about one half will set 
the second most significant bit to zero, too. Repeating this procedure we can 
set at most N bits to zero until running out of available partitions. To get 
a perfect partition with all b bits being zero, N must be larger than b. This 
consideration ignores the carry bits, which lead to the logarithmic corrections 
in eq. (36). 

What happens if iV < b resp. k > k c ? According to the approximative consid- 
eration above we expect the optimum partition difference to be exponentially 
small, 0(2~ N ), but larger than zero. It looks as if the zero temperature entropy 
is negative in this case. This is definitely wrong because the zero temperature 
entropy is by definition the logarithm of the number of ground states, which 
in any case is at least In 2. It turns out that we have to be more carefull with 
the limit T — > to get the correct zero temperature entropy. In terms of Aa 
the condition k > k c means 



TN > A i^ww) (37) 

i.e. essentially Aa = 0(2~ N ). In this regime the contributions of S are O(N) 
for any finite T, 

£(^) = ln(£) + l + 0(^), (38) 

hence cannot be neglected. Technically we deal with this contribution by in- 
troducing an effective "zero" temperature T below which the system can not 
be "cooled". T guarantees that the contribution of S remains O(N). Its value 
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Fig. 3. Average solution of the number partitioning problem with input numbers 
dj being i.i.d. uniform on [0, 1] compared to the analytical result Eq. (40) (straight 
line). Each data point is the average over 10 4 random samples. 

can be calculated from the lower bound of S: 

A T 1 

In 2 = N(k c - k) In 2 + S(—) « N(k c - «) In 2 + ln(-^). 

T Aa 



From that we get 

T = 2Aa2 Af(K ~ Kc) = ^2ttN (a 2 ) 2~ N . 
In this regime the average ground state energy (Ei) is no longer but 

(E,) = T = J2nN (a 2 ) 2~" 



(39) 



(40) 



This equation completes the previously known result that the average value 
of Ei is 0{yN 2~ N ) for real valued input numbers [18,23] by specifying the 



prefactor to be J2tt (a 2 ). 

To check Eq. (40) we consider the continous variant of number partitioning, 
where the a^ are real numbers, uniformely distributed in the interval [0, 1). In 
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Fig. 4. Phase diagram of the random number partioning problem. Nk is essen- 
tialy the number of significant bits to encode the input numbers, see Eq. (33). The 
squares denote the phase boundary found numerically. The solid line is given by n c 
from Eq. (32). For k < k c , the zero temperature entropy is extensive and a search 
algorithm typically finds quickly one of the 0(2 N ) perfect partitions. For k > k c , 
no perfect partitions exist, and the optimization problem has a hard to find, unique 
solution. 



our formalism this means Aa 
regime and Eq. (40) becomes 



and £ 



,■ a. 



N/3. We are in the k > K c 



En 



-irN2- N = 1.447 y/N 2~ N 



(41) 



In Fig. (3), Eq. (41) is compared to numerical data. The agreement is con- 
vincing. 

To check whether k(N) is a control parameter with a phase transition at k c (N), 
we did numerical simulations. For fixed N and k we calculated the fraction 
of instances that have at least one perfect partition. In accordance with Gent 
and Walsh [12] we find that this fraction is 1 for small k and for larger 
k. The transition from 1 to is sharp. Fig. (4) shows the numerically found 
transition points for 10 < iV < 28 compared to k c (N) from Eq. (32). Again 
the agreement is convincing. Note that K c (iV — > oo) = 1. The asymptotic 
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estimate 0.96 given by Gent and Walsh is probably due to the influence of 
the O(jjlogN) term in eq. (32) which cannot be neglected for system sizes 
accesible for simulations (N < 30). 

Before we turn to the constrained NPP, let us summarize what we have found 
so far: The statistical mechanics analysis of the NPP reveals two different 
phases, distinguished by the value of a parameter k, eq. (33), which corre- 
sponds to the number of significant bits in the encoding of the input numbers 
Oj divided by N. For k < K c , we have an exponential number of perfect par- 
tition, hence an exponential number of solutions to the NPP. For k > k c , we 
only have 2 solutions with a partition difference given by eq. (40). 



3. 3 Statistical mechanics of the constrained NPP 



The partition function of the constrained NPP is 

Z=J2'e-^i aj9jUT , (42) 

{Sj} 

where the primed sum denotes summation over all spin configurations with 
J2j Sj = mN. By now we know how to separate the absolute value on the 
exponent from the summation variables, but here additionally we have to get 
rid of the the constraint J2j s j — m N to do the summation. The discrete 
version of the Dirac 5 function, the Kronecker S symbol 



r n dm - , x 1 if to = ra 

Km = / — e* m( "- m) = , (43) 

J " K Z7T I if to ^ n 

for integer to, n can be used to achieve this: 

z = E ^. S] , m N^^ ajsA = r — e -^ mN £ e ^£^-l£-^l/T 

Sj} { Sj } 

The remaining sum can now be done exactly like in the preceeding section. 
The result is 

Z = 2 N r' 2 d -V r d ±e NG ^ (44) 

J-tt/2 TV J—k 2ll 

with 
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1 N a 

G(y, rh) = imm + — J^ In cos (-=7 tan y + rh) 

3=1 

pa imm + (In cos(— tan(y) + m) \ . (45) 

Compared to the unconstrained case we are left with a twofold integral and 
a complex valued integrand. The generalization of the Laplace method to 
complex integrands is the saddle point method [2,24]: Let the real part of 
G(y,m) have a maximum at (yo,m ). Then 



NG{y ^ ) dydmp* e ArG(yo ' Ao) dydme-^^ ^^ 



= e NG(y ,rh ) ^ = (46) 

for large N. G is the 2x2 Hesse matrix 

d 2 G(y,m) d 2 G(y,m) 

Q — \ d ' 2 y dydrh j r^j\ 

1 d 2 G(y,m) d 2 G(y,m) ' 

drhdy d 2 y 

where the derivatives are taken at the saddle point (yo,rho). I n our case the 
saddle point equations are 

= 8G ^ m > = ^tan(|tany + m)^ (1 +tan 2 y) (48) 



dG(y,m) I , a , A , . 

= ^ — - = (tan(— tany + m) > +im. (49) 

dm \ T I 

Like in the unconstrained case, we will assume that a can only take on values 
that are integer multiples of a fixed number Aa. With this assumption and 
the ansatz 

T 

tany k = kn— + iTx k = 0, ±1, ±2, . . . (50) 

Aa 



rh = —im (51) 
the saddle point equations simplify to 

(atanh(ax + rh)) = (52) 

(tanh(ax + rh)) = m. (53) 



For given value of m, the saddle point equations yield a solution (x, rh), which 
in turn gives rise to an infinite number of saddle points (eq. (50)). The con- 
tribution from all these saddle points have to be summed up to give Z, the 
groundstate energy E\ and the entropy S. Unfortunately, for m > we can 
solve the saddle point equations only numerically. Therefore we will for the 
time being concentrate on the balanced NPP, m = 0. In this case, the solution 
is trivial, x = rh = 0, the determinant of the Hesse matrix is 

detG= (a2) ~ 2 (a)2 ^l + ^A; 2 y k = 0, ±1, ±2, . . . . (54) 

With the help of eq. (27) it is straightforward to sum up the contributions 
from all saddle points. The result is: 

„ AT Aa , Aa 

Z = 2 N coth — (55) 

Niry/(a 2 ) - (a) 2 T 

The partition function for the balanced NPP is very similar to the one of the 
unconstrained NPP, eq. (28). Only the denominator changes from J%N (a 2 ) 



in the unconstrained to irNy (a 2 ) — (a) in the balanced case. The discussion 
of entropy and groundstate energy is very similar, too. The entropy can be 
written as 

S = N(k c - k) In 2 + S(Aa/T) (56) 

where S is the same as for the unconstrained NPP (eq. (34)) and the order 
parameter k and its critical value k c are 

K c = l-±ln 2 (-J=N) (57) 



^^M^(a 2 )-(a) 2 ). (58) 

The condition for the existence of perfect partitions, k < K c , translates into 

b<N-\n 2 (-^=N). (59) 



for input numbers aj being i.i.d. 6-bit integers (cf. eq. (36)). From fig. (2) one 
can tell, that for b = 25 N must be larger than 30 for perfect partitions to 
exist. Eq. (59) yields iV > 29.75. 

For k > k c , thee optimum partition difference E\ reads 

(E,) = ttvV) - (a) 2 -N-2- N (60) 
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Fig. 5. Average minimum residue of the balanced number partitioning problem with 
real numbers < aj < 1 compared to the analytical result Eq. (60) (straight line). 
Each data point is the average over 10 4 random samples. 

for the balanced NPP. Again this result fits very well with the numerics, see 
fig. (5). 



3.4 Overconstrained NPP 



For m > we solve the saddle-point equations (52) and (53) numerically. 
Fig. (6) displays the solution for input numbers a that are i.i.d. uniform over 

[0,1]. ' 

The solution diverges if m approaches a critical value m c = 0.41 .... For larger 
values of m, the saddle-point equations have no solution. This is no surprise: 
tanh() is a monotonic function with —1 < tanh() < 1. Eq. (52) requires that 
tanh(ax + fh) changes sign within the integration range. Therefore the right 
hand side of eq. (53) has to be smaller than 1. For input numbers distributed 
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Fig. 6. Solution of the saddle point equations (52) and (53) for input numbers a 
that are i.i.d. uniform over [0, 1]. 



uniformely over [0, 1] the saddle-point equations are 

/ da ata,nh( ay + in) = 
Jo 



(61) 



/ da tanh(ay + m) 
Jo 



m 



(62) 



and it is easy to show that \m\ has to be smaller than m c = \[2 — 1 = 0.41 . . . 
for a solution to exist. 

In statistical mechanics, a diverging solution often indicates that the properties 
of the system change drastically - a phase transition. What kind of phase 
transition is related to a critical value of the "magnetization" ml If m is 
close to 1, the NPP is over constrained, i.e. we cannot expect to find a perfect 
partition. Instead, the optimum partition is the one that collects the |(1 — m) • 
N largest numbers Oj in the smaller subset. Let a' be the |(1 + m) percentile 
of a, 



1 r a ' 

-(1 + m)— / p{a)da. 

2 Jo 

The partition difference E s of this sorted partition then reads 

ra' /-oo 

E s /N = / ap(a)da — / ap(a)da. 

Jo Ja' 



(63) 



(64) 



As long as E s > holds, the sorted partition is optimal. E s is positive if m 
is greater than a critical value m c that depends on the distribution p. For the 



uniform distribution on [0, 1], a' — \{1 + m) and 



E s /N= l l (l + mr- 1 -, 



(65) 
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which is positive provided 

m > m c = v2 — 1. (66) 

A more complicated derivation of Eqs. (65) and (66) using the statistical 
mechanics approach can be found in [7]. 

Note that the computational complexity of the overconstrained NPP (m > m c ) 
equals that of sorting N numbers, i.e. 0(N In N), so we have another phase 
transition from a computationally hard to a computationally easy regime in 
the NPP. 



4 The random cost problem 



The preceeding section has shown that the statistical mechanics of the NPP 
can be analysed rather easily. This is a remarkable exception. In general, spin 
glass models are much harder to deal with, and physicists have considered 
various simplifications. One of these simplified models was Derrida's random 
energy model, REM [5,6]. A cost function or Hamiltonian like Eq.(lO) maps the 
random numbers Jy onto 2 N random numbers E k , distributed according to a 
probability density p(E). Derrida's idea was to forget about the configurations 
{sj} and to consider directly the energies E k as independent random numbers, 
drawn from the probablity density p{E). The essential simplification, which 
leads to the analytic tractability of the model, is the assumption of statistical 
independence. 

The usefullness of the REM in spin glass theory has been discussed elsewhere 
[14]. Here we will concentrate on its counter part in combinatorial optimiza- 
tion, the random cost problem: Given are M random numbers E k , indepen- 
dently drawn from a density p{E). Find the minimum of these numbers. Since 
every number has to be considered at least once, the computational complex- 
ity of the random cost problem is O(M). The statistical independence of the 
numbers prevents an efficient heuristic: Any heuristic algorithm that considers 
only K <C M numbers is no better than simple sequential search through an 
arbitrary ^-element subset of the list. 

The motivation to study random cost problems stems from the fact that every 
combinatorial optimization problem with random inputs can be approximated 
by a random cost problem. If the original optimization problem has M feasible 
solutions and the costs of these solutions are distributed with density p{E), 
in the corresponding random cost problem we shall simply assume, that the 
M costs are drawn independently from p(E). For the NPP the approximation 
by a random cost problem gives apparently correct results at least for the 
statistics of the low cost configurations [27]. 
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4-1 Distribution of costs 



To find a random cost problem that corresponds to the NPP, we first have to 
calculate the probablity density of the costs. For the constrained NPP, p(E) 
reads 

p^ = (n + ) E'(^-IE^I)}- (67) 

where the primed sum runs over all configurations with iV + = iV|(l + m) spins 
Sj = +1. Since the numbers ctj are drawn independently from an identical 
distribution, the average in Eq. (67) depends only on N + , but not on the 
particular spin configuration. Ignoring the absolute value in the cost function 
for a moment, we may write 



^(£-E^)) = / 



dy gN~N+ (y) 9n+ (E + y), (6£ 



where gx is the probablity density of the sum Y,f=i a j- The central limit the- 
orem tells us that for large K 

1 ( (y-K(a)f\ , x 

where a 2 = (a 2 ) — (a) is the variance of a. Hence 

K^^^- (fl -'^/' (a>)2 )- ( ™> 

With 2N + — N = mN and taking the absolute value of the cost function into 
account we finally get 

/rnX Q(E) ( (E-m(a)N) 2 (E + m (a)N) 2 \ 

ME) = V^N V^^~ + e '^^) (71) 

as the probablity density for the costs in the random, constrained NPP. Q(x) 
is the step function, Q(x) = 1 for x > 0, Q(x) = for x < 0. 

To get the density of the costs for the unconstrained NPP, we have to sum 
over all values N + , 

N ( N\ 

p(E)=2~ N £ } Pm =2N + /N-l(E). 

N+=0 \ iV + / 

For large N, the sum is dominated by terms with N + = iV^4p, m = 0(1), 
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and we may apply the asymptotic expansion 

f N \ 2 N 



e -Ns(m) ( ?2 ) 



with 

1 + m 1 — m 

s(m) = ln(l + m) -\ ln(l — m), (73) 

and we may replace the sum over N + by an integral over m, 

p (E) = ^Lf 1 dme- Ns ^p m (E). 

V2-K J-i 



s{m) has a maximum at m = 0. Applying the Laplace method to evaluate the 
m-integral for large iV we finally get 

p(E) = 29(E) e'^W (74) 

^2tt (a 2 ) N 

as the probability density in the random unconstrained NPP. 
4-2 Statistics of the optimum 



We may now specify the random cost problem that corresponds to the NPP: 
Given are M random numbers Ei, independently drawn from the density p{E), 
eq. (71) resp. eq. (74). Find the minimum of these numbers. To connect to the 
NPP, M is chosen to be 

2\N(l + m)/2) ^N(l-m 2 ) { } 

for the constrained NPP and M = 2 N ~ X for the unconstrained NPP. Our claim 
is that the NPP is very well approximated by this random cost problem. 

Let Ek denote the k-th lowest cost of an instance of our random cost problem. 
The probability density p\ of the minimum E\ can easily be calculated: 

/ [Ex \ M-l 

Pl (E 1 ) = M ■ P(E 1 ) ■ M - / P(E')dE') (76) 

Ei must be small to get a finite right-hand side in the large M limit. Hence 

we may write 
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P i(Pi)«M.P(0)-(l-PiP(0)) 

^M-P(0)-e- MP ^ E \ 

This means that the probability density of the scaled minimal cost, 

ei = M ■ P(0) • Pi (77) 

for large M converges to a simple exponential distribution, 

Pl (e)=e- £ -e(e). (78) 

Note that a rigorous derivation from eq. (76) to eq. (78) can be found in 
textbooks on extreme order statistics [9]. Along similar lines one can show 
that the density p& of the k-th lowest scaled cost is 

k— 1 

ft(£) = FlJ!' e " £ ' G(e) A; = 2, 3,.... (79) 

Let us compare eqs. (78) and (79) with known analytical and numerical results 
for the NPP. From the moments of the exponential distribution Eq. (78), 
(e n ) — n\, we get 

r = ^ = 1 (80) 

for the relative width of the distribution, in perfect agreement with the numer- 
ical findings for the NPP [7]. The average minimal cost is (Pi) = 1/(M-P(0)), 
which gives 

(E 1 ) =n-a-N-2- N . e^ + « (81) 

for the constrained and 



(Pi) = ^2tt (a 2 ) -y/N-2- N (82) 



for the unconstrained case. Again this is in very good agreement with numer- 
ical simulations for the NPP [7] and in perfect agreement with our analytical 
results from the preceeding section, eqs. (40) and (60). For the constrained case 
with m > 0, the minimal cost increases with increasing m. This is reasonable, 
but nevertheless eq. (81) must be wrong for \m\ > 0. For input numbers a 
drawn uniformely from [0, 1] eq. (81) predicts that (Pi) is exponentially small 
as long as \m\ < 0.583 . . . , but we know from the preceeding section that 
for m > m c = \/2 — 1 = 0.414 . . . the NPP is overconstrained, hence has 
(Pi) = O(N). 
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Fig. 7. Distribution of the scaled optimum for the balanced number partioning 
problem. The solid line is given by Eq. (78), the symbols are averages over 10 
random samples. 

To check that the random cost approximation does not only give the correct 
first and second moment of the optimum of the balanced NPP, we calculated 
the distribution of E\ and higher energies numerically. Figs. (7) and (8) display 
the results for the balanced NPP. Equivalent plots for the unconstrained NPP 
look similar. The agreement between the numerical data and Eqs. (78) and 
(79) is convincing. 

How can the random cost problem be so similar to the NPP? The answer 
is, that there is in fact a certain degree of statistical independence among 
the costs in the NPP. In the appendix we show that the joint probability 
p(E, E') factorizes, i.e. p(E, E') = p(E)p(E') for the unconstrained and the 
balanced NPP, but not for the constrained NPP with \m\ > 0. This is a 
necessary, but not sufficient condition for independence, but the approach can 
probably extended to a complete proof of independence. Here we adopt a 
physicists attitude and consider the random cost problem to be a very good 
approximation to the NPP. 
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Fig. 8. Distribution of scaled k-th lowest energy for the balanced number partioning 
problem. The solid lines are given by Eq. (79), the symbols are averages over 10 5 
random samples of size TV = 24. 

4-3 Poor performance of heuristic algorithms 



The correspondence between the NPP and the random cost problem not only 
provides analytic results on the NPP but also has some consequences for the 
dynamics of algorithms: Any heuristic that exploits a fraction of the domain, 
generating and evaluating a series of feasible configurations, cannot be signif- 
icantly better than random search. The best solution found by random search 
is distributed according to Eq. ((76)), i.e. the average heuristic solution should 
approach the true optimum no faster than 0(1/ M), M being the number of 
configurations generated. Note that the best known heuristic, Korf's CKK 
[21,26] converges slower, namely, like 0(1/M a ) with a < 1 to the true opti- 
mum. Other heuristics, like simulated annealing, are even worse [16]. 

The random cost analogy means that there is hardly any correlation between a 
partition and its cost. Partitions that are similar to each other may have very 
different costs and vice versa. One might argue that this picture depends on 
the encoding of a partition, especially on the precise definition of "similarity" . 



27 



E 

E 

-4— ' 

Q. 
O 



O 

c 

CD 
=3 
CX 
CD 



CD 



0.5 



0.4 



0.3 



0.2 



0.1 









01 23456789 10 11 12 
number of right branches in search tree 



Fig. 9. Distribution of optima in the search tree spanned by the CKK algorithm. 
The number of right branches, the discrepancy, is a measure of distance to the 
heuristic Karmarkar-Karp solution. The NPP solved was unconstrained, with input 
numbers drawn uniformely from [0, 1]. The number of instances is 10 for N < 20 
and 10 3 otherwise. 

Throughout this paper we used the obvious encoding of a partition as a set 
of binary variables and one can show that indeed partitions with similar costs 
are completely dissimilar in terms of a vanishing overlap, -^ J2j s j s 'j — (see 
appendix). 

Maybe there is a better problem representation for the NPP, an encoding that 
centers the good solution around a known position in search space. In fact it 
has been found that the choice of encoding is more important than the choice 
of search technique in determining search efficacy [32]. None of the proposed 
encodings and search techniques is more efficient than Korf 's CKK, however. 

An ansatz to concentrate good solutions in a small part in search space was 
proposed by Korf [21]. In the tree spanned by the CKK, a left branch follows 
the Karmarkar-Karp differencing heuristic, a right branch does the opposite. 
Instead of searching the tree depth first like in the CKK one might as well 
search the paths of the tree in increasing order of the number of right branches, 
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Fig. 10. Discrepancy of the optimum partition: Mean (left) and standard deviation 
(right) vs. N from the data of fig. (9). The fit of y/N — 4 for the standard deviation 
takes into account that the actual search tree is only explored to depth N — 4, since 
for N < 4 the Karmarkar-Karp solution is always optimal [13]. 

or discrepancies, from the heuristic recommendations. The hope behind this 
limited discrepancy search [15] is of course that good solutions are "close" to 
the Karmarkar-Karp heuristic solution. 

To check whether the discrepancy is a good measure to search the optimum, 
we calculated the discrepancy of the optimum partition numerically. Figs. (9) 
and (10) show that for large N, the discrepance of the optimum is a Gaussian 
distributed random variable with mean oc N and variance oc y~N. Hence a 
search guided by small discrepancies is not significantly better than simple 
random search and our random cost analogy persists in discrepancy space. 
Note however that there are some correlations between the optimum and its 
discrepancy. For true random discrepancies we would get an average of |iV, 
but from fig. (10) we get a smaller value ofabout jqN. This is no surprise: 
the partition with the highest discrepance N — 1 has maximum cost, and 
partitions with high discrepancy are never optimal. Hence the relevant range 
of discrepancies is not N but a fraction xN, x < 1. The best a heuristic search 
can do is to avoid the very large discrepancies. Note that our independent cost 
assumption does not hold for costs 0(N) (see appendix). 



5 Summary and conclusions 



The main contribution of this paper is the application of methods and ideas 
from statistical mechanics to the number partitioning problem. The typical 
computational complexity of the NPP undergoes a sudden change if the sys- 
tem size and/or the number of bits in the input numbers is varied across a 
critical value. The standard statistical mechanics approach yields a quantita- 
tive, analytic theory of this phasetransition. Furthermore, it reveals a second 
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phasetransition in the constrained NPP, controled by the imposed cardinality 
difference m. 

The idea of a random energy model, born in spin glass theory, can be refor- 
mulated as random cost problem in combinatorial optimization. Numerical as 
well as analytical results support our claim, that the balanced and the uncon- 
strained NPP are extremely well approximated by a random cost problem, at 
least if one excludes the very high costs. On the one hand this correspondence 
is responsible for the bad performance of heuristic search algorithms but on 
the other hand allows to derive new anayltic results like the probability density 
of optimal and suboptimal costs. 

What contributions does this work make beyond the specific problem of num- 
ber partitioning? First, it provides an example of the applicability of statistical 
mechanics to combinatorial optimization that is exceptionally simple. It may 
well serve as a pedagogical introduction into this interdisciplinary field. Second 
the idea of the random cost problem may have applications beyond number 
partitioning. The dynamics of heuristic algorithms for other combinatorial op- 
timization problems can be checked for a signature of a corresponding random 
cost problem, possibly with a differing p(E). 



A Evidence for the independent cost assumption 

Let p(E, E') denote the joint probability density of having costs E and E' in 
the same instance of an NPP, 

p(e,e?) = (") e'e'^-ie^d-^'-ie^dV (a- 1 ) 

V V +/ { S] }{s>} \ 3 3 I 

If the costs were independent, this probabilty density should factorize, p(E, E') - 
p(E) ■ p(E'). This is what we are going to check in this section. Our line of 
reasoning is similar to the proof of theorem 4.9 in [4]. 

Consider the quantity 

p N++ (E, E') := (S(E - E%^) • W - E%4)) • ( A - 2 ) 

All indices j are treated equally in the average over the a,j, hence p can depend 
on {sj} and {s'j} only through the number N ++ of spins Sj = s'j = +1. Hence 
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we can write 

• (p N++ (E, E') + p N++ (-E, E') + p N++ (E, -E') + p N++ (-E, -E')) (A.3) 

The four p terms and the G-functions take into account the absolute value 
of the cost-function which we have omitted in the definition of p. There are 
( N J possible ways to choose the +1 spins in {sj}. This factor cancels one 
of the normalization factors in eq. (A.l). Among the N + +1 spins in {sj} we 
can choose N ++ spins that are +1 in {s'A, too. The remaining N + — N ++ +1 
spins in {s'A can be chosen among the N — N + spins that are —1 in {sj}. This 
yields the two binomial factors in eq. (A.3). Let 

A = {j:sj = +1} A = {j : s 3 ; = -1} 



A = {j : s' 3 = +1} A = {j : a'j = -1} 
be the partitions corresponding to both spin sequences. With 

Vi-.= E a 3- E_% ( A - 4 ) 

jeAnA' jeAnA' 



V 2 := J2_ a J- E a J ( A - 5 ) 

j&AnA' jeAnA' 



we can write 



N N 

E a jSj = V l + V 2 E a J s 'j = v i- V* ( A - 6 ) 

3=1 3=1 

Now the nice thing about V\ and V 2 is that they depend on two disjoint subsets 
of the numbers a,, hence are statistically independent. Let p\ and p 2 denote 
the probabilty density of V\ resp. V 2 . We may write 

Pn ++ = \-Pi(\l E + E '\) ■ P2 (\[E ~ #']) • (A.7) 

V\ and V 2 are composed of sums of independent random numbers. The first 
sum in V\ runs over all elements that have Sj = s' = +1, the second over 
those with Sj — s'. ■ — — 1. Counting the number of elements in these sets, 
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\AnA'\ 


= N ++ 


\AnAt\- 


= N+- N ++ =: N + _ 


\AnA'\ 


= N+- N ++ =: JV_+ 


{AnA'l 


= N- 2N+ + N ++ =: N. 


we can write 





pi(V 1 )=J dzg N++ (z + V 1 )g N __(z) (A.8) 

p 2 (V 2 ) = fdzg N+ _(z + V 2 )g N _ + (z), (A.9) 



where g% is the probabilty density of the sum of k numbers a,j. As argued 
above, pN ++ (E,E') depends on the spin sequences only through N ++ , 

p N++ (E, E') = ^J dz g N++ (z + l - [E + E']j g N __ (z) 

■ j dzg N+ _ (z + hfi-E^j g N _ + {z) (A.10) 



Now we consider the large N limit. We are interested in the case m = 0(1), 
hence N ++ = 0(N) and the sum over N ++ in eq. (A. 3) is dominated by 

contributions with N ++ = O(N). It is convenient to express N ++ , N , N_ + 

and N + _ in terms of the overlap parameter 

1 N 



N J=i 



which is 0(1) in this scaling regime: 



N N 

N ++ = -(l+m)--(l-q) 

N, , N, 

N.. = -(l + m)--(l-q) 

N_ + = N + _ = ^(l-q). 



Approximating all distributions g in eq. (A. 10) by their asymptotic expansions, 
eq. (69), we get 

1 ( (E - (a) mNf (E> - (a) mNf \ 

Pn ++ {h,,h, ) — - — 9A . ,, ,f exp 



2na 2 N^/l^q 1 *\ 2a 2 N{l-q 2 ) 2a 2 N{l-q 2 ) ) 

expf AE + E')(a)mN-EE'-(afm^ \ 
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Note that Pn ++ (E, E') factorizes only for q — 0. 

In the scaling regime m = 0(1), q = 0(1) and N large, we can replace the 
binomials in eq. (A. 3) by their asymptotic expansions according to eq. (72) 
and the sum over iV ++ by an integral over q. This integral in turn can be 
calculated asymptotically using the Laplace method. As a matter of fact the 
product of the binomials, 

AT \ /N(t _ ^\\ /N(i | ^\\ 

(A.13) 

has a maximum at q — m 2 , hence we expect the factorization only for m — 0, 
the balanced NPP. In fact, in this case we get 

^ I dq -e- N ^-.. (A.14) 






2tt J 1-q 

with s(q) from eq. (73), which has a maximum at q — 0. This proves that 
p(E, E') factorizes asymptotically for the balanced NPP. Note however, that 
even for m = the saddlepoint is not at q — if E, E' — O(N). Costs this 
large are not independent. 

For the unconstrained NPP the calculation of p(E, E') is a bit more cumber- 
some, but similar. Here we have two additional integrals over m = -^ J2j s j an d 
m' — jj J2j s'j, but in the limit A^ — > oo the major contributions come from the 
saddle point at m — m' — 0. Hence we observe the asymptotic factorization 
for the unconstrained NPP, too. 
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